Somewhat Stochastic Matrices

The notion of a Markov chain is ubiquitous in linear algebra and probability books. In linear algebra a Markov chain is a sequence {xk} of vectors defined recursively by a specified vector x0, a square matrix P and the recursion xk = Pxk−1 for k = 1, 2, . . .. That is, xk = P x0. Natural probabilistic restrictions are imposed on x0 and P . It is assumed that x0 is a probability vector; that is, its entries are nonnegative and add up to 1. It is assumed that P is a stochastic matrix; that is, it is a square matrix whose columns are probability vectors. The original version of the main theorem about Markov chains appears in Markov’s paper [2]. In the language of linear algebra it reads: Suppose that P is a stochastic matrix with all positive entries. Then there exists a unique probability vector q such that Pq = q. If {xk} is a Markov chain determined by P , then it converges to q. More generally, the same conclusion holds for a stochastic matrix P for which P s has all positive entries for some positive integer s. All elementary linear algebra textbooks that we examined state this theorem. None give a complete proof. Partial proofs, or intuitive explanations of the theorem’s validity, are always based on knowledge about the matrix’s eigenvalues and eigenvectors. This argument becomes sophisticated when the matrix is not diagonalizable. What these proofs leave obscure is a certain contractive property of a stochastic matrix already observed by Markov. Of course, this contractive property is explored in research papers and some advanced books. However, the relative simplicity of the underlining idea gets lost in the technical details of an advanced setting. We feel that this contractive property deserves to be popularized. We use it here to provide a direct proof of a theorem which is more general than the one stated above. We consider real square matrices A whose columns add up to 1. Such a matrix we call a somewhat stochastic matrix. The probabilistic condition that all entries be nonnegative is dropped. Instead of assuming that all entries of A are positive, we make an assumption about distances between the columns of A. This assumption leads to a contractive property of a matrix that yields convergence. This and other definitions are given next.